Riemannian game dynamics

We study a class of evolutionary game dynamics under which the population state moves in the direction that agrees most closely with current payoffs. This agreement is defined by means of a Riemannian metric which imposes a geometric structure on the set of population states. By supplying microfoundations for our dynamics, we show that the choice of geometry provides a state-dependent but payoff-independent specification of the saliences of and similarities between available strategies. The replicator dynamics and the (Euclidean) projection dynamics are the archetypal examples of this class. Similarly to these representative dynamics, all Riemannian game dynamics satisfy certain basic desiderata, including positive correlation and global convergence in potential games. Moreover, when the underlying Riemannian metric satisfies a Hessian integrability condition, the resulting dynamics preserve many further properties of the replicator and projection dynamics. We examine the close connections between Hessian game dynamics and reinforcement learning in normal form games, extending and elucidating a well-known link between the replicator dynamics and exponential reinforcement learning.